Substantive Validity of a Simulation-based Game

Games are seen as attractive potential learning tools because of their ability to engage players and immerse them in situations that invite application of content and skills. However, it is not always clear whether educational games encourage players to utilize particular cognitive processes, access appropriate prior knowledge concepts, and/or apply given procedural skills as intended. Research of substantive validity examines evidence of the cognitive processes students use to complete tasks. This paper examines evidence for the substantive validity of a simulation-based game using recorded sessions of eleven pairs of students (six beginning and five advanced) playing the game. Evidence for the use of troubleshooting skills, prior knowledge, and procedural skills was analyzed. Cognitive processes related to troubleshooting were revealed and differences in the processes of beginning and advanced students were suggested by the data. These findings are discussed in the larger context making inferences about student performance using educational games.